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ABSTRACT 

Michigan Literacy conducted a project to collect, 
review, summarize, and compile a reference document of all student 
assessment tools currently being used by adult volunteer literacy 
programs in Michigan. Data were gathered through a survey 
questionnaire that was sent to 134 volunteer literacy programs (with 
82 responses) . Respondents reported that they used 40 different 
assessment instruments to determine suitable materials for students, 
to place students appropriately, to assess student progress, to make 
recommendations, and to identify student areas of difficulty. In 
addition, relevant literature was reviewed to identify current 
research and development on adult student assessment tools 
specifically designed for use in volunteer literacy programs. A 
resource list and the summary of the literature were provided to 
assist programs in identifying student assessment methods best suited 
to their students' and their programs' objectives. Eleven assessment 
tools were noted as being used in various Michigan programs, with 
contact persons and telephone numbers given for each. (The report 
includes the following: an annotated bibliography describing 10 
resources, two ERIC Digests on assessment, and a list of assessment 
tools used by respondents. Contains 10 references.) (KC) 
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SUMMARY OF THE PROJECT 



The Adult Volunteer Literacy Student Assessment Study was funded 
by grant funds from Section 107c of the State School Aid Act, and 
administered by the Office of Workplace Development of the Michigan Jobs 
Commission. The goals of the project as proposed by Michigan Literacy, 
Inc. were first to collect, review, summarize and compile a reference 
document of all student assessment tools currently being used by adult 
volunteer literacy programs in Michigan. Secondly, a review of the 
literature was conducted to identify current research and development on 
adult student assessment tools specifically designed for use in volunteer 
literacy programs. The provision of the resource list and the summary of 
the literature are intended to assist programs in identifying student 
assessment methods best suited to their students' and their program 
objectives. 

To collect the information on instruments currently being used in 
Michigan, a survey questionnaire was sent to 134 volunteer literacy 
programs. The questionnaire also asked for the name and phone number of 
a contact person from each agency. Concurrently, an extensive search of 
the literature was conducted which actually turned up very little in the 
way of specific instrumentation for adult literacy assessment used in 
volunteer literacy programs. Not surprising, much of what has been 
developed and used for adult basic education has been borrowed by the 
volunteer literacy field with reasonable success. 

In addition to the above, the project also convened an advisory 
committee of professionals to identify what they perceived to be the 
strengths and challenges associated with the most frequently used 
assessment instruments as reported by the literacy community. The 
committee reviewed the preliminary data and summarized their thoughts 
and recommendations for inclusion in the section of this report entitled, 
"Data Summary and Recommendations". The advisory committee members 
were chosen because of their expertise in the literacy field, and for the 
diversity of location and types of programs with which they work. 

It was not the intent of the project to evaluate on in any other way 
judge the effectiveness of adult volunteer literacy programs. The intent 
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was rather to compile this information and distribute it to literacy 
providers so that they can perhaps address for themselves the adequacy of 
their program's student assessment practices. The summary of the 
literature and the bibliography will enhance the practitioner's ability to 
critically choose what might work best for their program. It is further 
intended that the ready availability of this reference document will 
provide the needed incentive for those who have not yet begun a formal 
assessment program, to contact one of their coileagues and discuss how 
to get student learner assessment started in their program. 



REVIEW OF THE LITERATURE 



Assessment is to educational and institutional 
goals as maps are to travelers and their destination. 
(Rickard, Stiles, et.al., 1991) 

Assessment and education do seem to go hand-in-hand even in the 
adult volunteer literacy field. We use the same word, assessment, to 
refer to several distinct activities, and most would agree it includes 
much more than testing (BCEL.1990). Rickard and Stiles (1991) describe 
learner assessment activities as including the means to place students 
into program levels, to prescribe specific learning plans, to monitor 
student progress and to certify competency. Often programs rely on "off 
the shelf" general literacy tests, many of which have been normed on the 
average performance of children at various grade levels (BCEL.1990). 

There is very little in the current literature that discusses recently 
developed and validated instruments that are appropriate for the adult 
engaged in a tutoring relationship to improve their literacy skills. The 
literature supports that the Tests of Adult Basic Education (TABE), the 
Adult Basic Learning Exam (ABLE) and the California Adult Student 
Assessment System (CASAS) are the most frequently used tests for 
student assessment in adult literacy programs nation-wide, and all three 
of these instruments are adult in content. Most current thinkers on the 
subject of literacy assessment are proposing a shift away from 
standardized assessment and more toward competency-based or 
participatory assessment methods in which the adult learner's goals 
become the focus of the assessment strategies. 

Though the adult education field is coming to recognize the 
limitations of existing standardized tests and that there are very few 
meant specifically for adults, it also appears that there is a hesitancy to 
use those designed with the adult student in mind (Metz,1990) and the 
hesitation is probably wise. Sticht (1990) recommends avoiding testing 
when an adult first shows up for a program. If too much time is spent on 
testing at the outset, adults may just drop out (Frager,1991) or perform 
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poorly just because they don't have test-taking skills. The process leads 
to low performance and high anxiety on the part of the student (Oakley, 
1988). Standardized tests are intimidating, may remind students of 
former failure and give a one-sided view of a multi-sided problem. They 
often do not measure anything pertaining to the goals of the student (Metz, 

1990) . 

Lytle and Wolfe. (1989) maintain that contrary to popular opinion, 
adult new readers often don't fit the image of dependent, weak failures. 
Instead, they operate "...within a complex social network in which they are 
interdependent, offering skills of their own in exchange for the literacy 
skills of others." Just as no single teaching strategy is effective for all 
students, there is no single assessment instrument or process that will 
provide quality information for all adult learners (Rickard and Stiles, 

1991) . 

Lytle and Wolfe (1989) describe four approaches to learner 
assessment. The first is the standardized approach which has been 
discussed above. This usually includes grade-equivalent scores, is cost 
effective, is usually independent of the curriculum, and measures literacy 
as reading skills. Materials-based assessment is commercially available 
and is related to progress in pre-determined materials, such as the 
assessments that are part of the Challenger Series by Laubach. 
Competency-baseo assessment is specified to real life tasks, has a 
continuum of difficulty, is administered frequently with feedback being 
provided, and includes a broad range of strategies. Lastly, there is 
participatory assessment. This includes literacy as practices and critical 
reflection, takes into account a range of texts, tasks and contexts, 
involves the learner as an active partner in the assessment, is ongoing and 
also provides a broad range of strategies. 

Richard Venezky (1992) suggests that there should be two paths that 
adults involved in literacy or basic education could choose from: that 
which leads to a GED or other formal education and that which gives the 
student practical skills they need for "self esteem, work, home and civic 
needs". If we agree that the standardized test is more closely associated 
with formal educational settings then it is expected that more 
standardized tests will be used for adults in ABE and GED classes. Within 
the structure of the ABE classroom, just like in the less formal 



tutor/student partnership, the adult learner brings a variety of goais and 
along with them, the need for a variety of assessment strategies to 
address them. 

Almost all of the literature of the 1990's on the subject of adult 
learner assessment, including literacy education, supports diversity in 
initial assessment as well as in on-going evaluation of student progress. 
It is not enough to just measure success with tangible literacy skills but 
we must also take into account the very real but difficult to measure 
outcomes such as self-esteem and the accomplishment of lifelong 
learning goals. In a paper to the Adult Literacy Assessment Workshop in 
Philadelphia in 1991, Rickard suggests that "good" assessment strategies 
must articulate with multiple indicators used to mark progress toward 
the goals of the learner, the goals of the program and the goals of the 
policy makers. 

Current publications identify the other purposes for assessment that 
go beyond the student placement and progress issues. Balancing the 
students' needs for success while demonstrating to stake holders that 
programs have successfully used their resources to achieve their goals is 
a complex task (Lytle and Wolfe, 1989). Program centered assessment 
evaluates how well a program meets the needs of its adult learners and 
also assesses staff training along with the availability and allocation of 
pertinent resources (Rickard, 1991). It would be a mistake to use 
measures of student progress as the only data for evaluating program 
successes (BCEL.1990). Administrators often want assessments that can 
be used before and after tutoring and which have the added feature of 
validity and reliability often found in the standardized instruments 
(Frager,1991). Though this is a part of the program evaluation, student 
progress alone may be due to any number of factors, some of which may be 
outside the influence of the program on the student. 

The continuing theme that most of the authors cited here speak to is 
the need for a multitude of approaches to meet the variety of student 
goals, the variety of activities we label as assessment, and the variety of 
audiences interested in the results of the assessments (students, tutors, 
administrators, funders. bosses, etc.). Though a fair amount of space has 
been devoted to the limitations of standardized tests, both those normed 
on grade-level (children) reading and those developed specifically for 
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adults, it is not the intent of this summary to lead the reader to conclude 
*hat all standardized tests should be banned from use. Instead, the 
consensus of the literature reviewed seems to be to not necessarily 
eliminate but to look beyond the standardized instruments; to build a 
montage of assessment strategies for adults involved in literacy 
programs that will take into account their unique and multifaceted needs 
for measures of success, however they may define that success. 

In conclusion, the following excerpt from the January, 1990 issue of 
the Newsletter For The Business C ommunity , summarizes well the most 
current and prevalent thinking on what assessment should be in the adult 
literacy field: 

A growing number of practitioners around the country agree and have 
begun to explore alternative approaches to assessment. The perspective 
that guides these efforts is that the paramount purpose of assessment 
should be to help the learner achieve his or her goals; that what is 
assessed must reflect what the learner wishes or needs to accomplish; 
that the process must build on the learner's experience and strengths 
rather than deficits; that assessment is not something done t£L the 
learner; that it should not be externally imposed nor shrouded in mystery, 
nor separated from what goes on in the regular course of learning activity. 
Rather, it is postulated, assessment should be an organic part of the 
learning experience - an ongoing collaboration between the teacher, the 
learner, and the text, to review and refocus what should take place in the 
light of progress being made. It should not depend on a single procedure 
but a variety of procedures. And one of its major functions should be to 
produce feedback that will make programs more effective. Most of all, 
testing instruments should convey respect for learners. 



DATA SUMMARY AND RECOMMENDATIONS 



Student assessment in literacy programs is conducted for many 
different reasons using many different methods. At intake, assessment is 
intended to result in information that will help programs choose 
materials that are suitable for the student, and to provide information to 
the tutor regarding the student's learning abilities, style and current 
reading level. 

Eighty-two of the 134 volunteer literacy programs in Michigan 
responded to the survey on student assessment and 76 of these reported 
conducting student assessments on a regular basis. Nearly all of the 
responding programs (92%) reported that they use assessment information 
to "determine suitable materials" or "for student placement". 

Once tutoring gets underway, student assessment is valuable as a 
means of measuring progress with reading as well as progress toward 
other life goals important for the student to achieve as a result of being 
involved in a tutoring relationship. Some programs do assessment when 
the student leaves the program to be able to document change from intake 
to termination. Of the programs responding, 87% reported doing student 
assessment before tutoring begins, 76% do assessment during the tutoring 
process and 32% complete assessments when students leave the program. 

The following list, in order from most frequently reported (by 38 
programs) to least frequently reported (by 1 program), cites the various 
ways programs use the assessment information they collect: 

□ to determine suitable materials for the student 

□ to appropriately place the student 

□ to assess student progress 

□ to make recommendation's to tutors 

□ to choose an appropriate tutor for the student 

□ to determine student goals 

□ to provide information to funders and the Michigan Department of 
Education 

□ to identify student study skills 

□ to make progress information available to students and tutors 

□ to provide baseline data 

□ to identify student areas of difficulty 




□ for program planning 

□ to determine program effectiveness 

The materials used to conduct these assessments vary as widely as 
the types of programs there are across the state. It is the consensus of 
this project that diversity in assessment practices is positive and should 
be encouraged so that programs continue to find means for collecting 
information about their students' initial levels of functioning and 
progress as they proceed with a tutoring program. It is unlikely that any 
single instrument or method will meet every program's need. 

Besides the important information that programs and tutors and 
students can learn about their efforts when assessment is done as a 
regular part of the program, there are also advantages to having aggregate 
assessment information available for audiences external to the program. 
Often funders and the Department of Education will require information 
about the progress made by participants in a program. Being able to report 
student goal achievement and/or reading improvement as documented by 
the program's assessment data, provides the concrete information that 
potential funders, volunteers and even potential students want to know 
before they commit their involvement to a program. No organization or 
individual for that matter, wants to assign their resources or energy to a 
program that has no track record of success. An ongoing assessment 
system will feed information into a volunteer literacy program's outreach 
and fund raising efforts that is far more compelling than trying to 
convince potential supporters of the worthiness of the program's efforts 
without documented proof of student progress. 

The volunteer literacy programs in Michigan which responded to the 
survey collectively use 40 different assessment instruments. 
Additionally, 24 programs reported having designed their own assessment 
tools and 38 programs employ open-ended interviews with their students 
as part of their assessment, strategies. Almost all programs use a 
combination of tools and methods to conduct student assessment, and as 
reported previously, conduct assessments at various times throughout the 
student's enrollment in the program. 

In considering the various tests most often reported to be used by 
Michigan programs, the Advisory Committee used their professional 
judgement to identify the following strengths and limitations: 



s 



1 INSTRUMENT 


STRENGTHS 


LIMITATIONS 


Slosson Oral Reading Test 
(Sort) 


• Quick 

• Provides a good starting point 

• Easy to administer 

• Inexpensive 

• Can be administered by 

volunteers 


• Lay-out can be 

intimidating 

• Words are out-dated 


Where-To-Start 
(from LITSTART) 


• Quick 

• Not intimidating 

• Good as an ice-breaker 

• Can be administered by 

volunteers 

• Diagnostic for word recognition 
and sounds, word attack skills, 
reading level indicator (0-4), 
and spelling skills 


• Works for initial 
assessment only 


Tests of Adult Basic Education 
(TABE) 


• Shows grade level and skill 
mastery 

• Fairly reliable and validated 

• Recent development 


• Not generally for 
volunteers to 
administer 

• Intimidating 

• Not for low-level 
readers 

• Cumbersome 

• Timed 


Botel Reading Inventory 
Word Opposites Sub Test 


• Quick 

• Can be administered by 

volunteers 

• Easy to score 


• Concept of opposites 
is hard 

• Evidence of cultural 
bias 



Portfolio assessment was mentioned specifically by one program. 
Though portfolios are commonplace in high schools and the junior high 
grades in Michigan, they are just beginning to be used in the adult 
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education arena. Portfolios are usually files or folders that contain 
collections of a student's work. They furnish a broad portrait of 
individual performance, assembled over time. As students put together 
their portfolios, they must evaluate their own work, a key feature of 
performance assessment. Portfolios are most common in the subject 
areas of English and language arts, where drafts, revisions, works in 
progress, and final papers are typically included to show students' 
development. Vermont and Michigan are among the states taking the lead 
on portfolio use for assessment (Rudner,1991). 

The Advisory Committee strongly recommends that volunteer 
literacy programs consider the portfolio assessment for implementation 
with their adult students. It appears to be the best method for ongoing 
assessment of student progress. Any standardized tests used at intake 
can become a part of the student's portfolio but the process additionally 
affords the opportunity to measure grovrth in many life areas in addition 
to reading level. Another advantage is that the portfolio assessment 
process is student-centered and therefore allows for student goals to 
become the yardstick along which a program, a tutor and a student can 
measure success. While not a lot of literacy programs have used portfolio 
assessment there are resources available through the State Literacy 
Resource Center and a few experienced practitioners within the state. 

In summary, it is a credit to the field of adult volunteer literacy in 
Michigan that so many programs (at least 58%) are involved in some 
method of ongoing student assessment. There are a variety of methods 
being employed and programs are encouraged to design an assessment 
process that best meets the needs of the students, tutors and program 
staff. A student-centered, multi-dimensional assessment such as 
portfolios is recommended for consideration to be used with adult 
students. There are a variety of reasons to do ongoing assessment, 
including to establish a baseline at intake, to be able to design an 
.appropriate program based on a student's needs and abilities, to measure 
progress throughout the tutoring process, and to make available concrete 
information about a program's successes for volunteer and student 
recruitment, and for fund-seeking ventures. Any programs beginning a 
student assessment process, or considering changing or expanding their 
current assessment practices, are encouraged to use this resource 



directory to identify and then contact their colleagues around the state to 
explore and exchange ideas about student assessment practices. 
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Literacy Practitioner. Assessment Issue. Literacy Volunteers of America, 
New York State, Inc. Vol. 2, No.1, August, 1994. 
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ADULT LITERACY LEARNER ASSESSMENT 



Learner assessment, the process of collecting and analyzing 
data provided by learners in order to make judgments about 
the literacy accomplishments of individuals or groups, is a key 
feature of adult literacy programs. Learner assessment occurs 
in different forms throughout an adult's participation in a lit- 
eracy program. It frequently reflects different views of literacy 
and learning and yields distinct types of information to different 
stakeholders. It provides information to teachers for use in 
instructional planning, to learners for determining their pro- 
gress toward particular goals, to program managers and staff 
lor evaluating the impact or instruction, and to hinders for 
establishing some degree of program accountability and success 
(Lytic and Wolfe 1989). 

Four major types of approaches to learner assessment have 
been identified in the literature: standardized testing materials 
based, competency based, and participatory. This ERIC Digest 
provides an overview 01 these four assessment approaches, 
including some issues affiliated with each. It ends with some 
suggested guidelines for selecting assessment procedures. 



Approaches to Learner Assessment 
Each of the four approaches to learner assessment described * 

1 .„ - ~ H - . ! ► philoSOp 1 ^"" 1 ~-:--'^''^-<- narfn fl M;, )a c 

literacy, 
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here reflects varying philosophical orientations and perspectives 
related to learners, literacy, and educational contexts. 



gories: their intrinsic defects and their misuse" (p. 6). The 
major intrinsic defect is the fact that they rely on grade-level 
equivalents, i.e., they have been normea on children. Such 
measures <fo not reveal the extent of the life experiences and 
knowledge that adults bring to an instructional program nor do 
they provide data that can be used in developing an appropriate 
instructional program. Other difficulties in the use of standard- 
ized tests involve the relationship of the tests to a program's 
instructional model and the fact that many adults associate 
them with previous school failure (BCEL 1990; Lytle and Wolfe 
1989). 

The misuse of standardized tests relates to the practice of 
employing them as the sole component of program evaluation. 
Although learner assessment is an important component of 
program evaluation, a number of other elements such as pro- 
gram management, teaching, and curriculum need to be exam- 
ined in judging program effectiveness (BCEL 1990). 

Improvements that address some of their intrinsic defects are 
being made in standardized tests. The Degrees of Reading 
Power test uses cloze passages and therefore reflects more 
current views of the reading process as the construction of 
reading. Item response theory, a psychometric theory that 
takes into account certain factors sucn as item difficulty, is also 
being applied in some standardized tests (Lytle and Wolfe 
1989). 



Standardized Testing 

Because standardized tests are relatively easy and inexpensive 
to administer, standardized testing is the most Nvidely used 
approach in adult literacy assessment in the United States. 
Large groups of adults can take a test under the supervision of 
a comparatively small number of administrators. In addition, 
the training requirements to administer the test are minimal 
(ibid.). 

By definition, a standardized test is designed to be given under 
specified, standard conditions. If it is not, the results are 
invalid (Business Council for Effective Literacy 1990j Sticht 
1990). Standardized tests may be either norm- or criterion- 
referenced. Many of the standardized tests of reading used in 
adult literacy programs are norm-referenced, that is, they 
measure an individual's performance against a "normal" perfor- 
mance established by others who have taken the test (BCEL 
1990; Lytle and Wolfe 1989). Criterion-referenced tests, on the 
other hand, d^sscss a learner's achievement against an absolute 
standard or criterion of performance rather than against a 
norming group (Sticht 1990). 

Despite their extensive use in adult literacy assessment, 
standardized tests have a number of critics among researchers 
and practitioners. According to the BCEL (1990|, the "objec- 
tions [to standardized tests] tend to fall into two broad cate- 



Materials-based Assessment 

Materials-based assessment refers to the practice of evaluating 
learners on the basis of tests following the completion of a 
particular set of curriculum materials. It shares some features 
with standardized tests such as availability through commercial 
publishers, ease of administration, and a view of literacy as 
reading skills. 

Although the materials-based approach to assessment makes 
possible a close connection between curriculum aiid assessment, 
it creates a closed system that does not invite analysis of 
teaching processes and materials. Because most of the curric- 
ulum is prepackaged, there is little opportunity for learners to 
direct their own study. Also, the literacy activities beyond the 
system go unassessed and may not be recognized as meaningful 
by learners and teachers (Lytle and Wolfe 1989). 

Competency-based Assessment 

Closely related to criterion-referenced standardized testing, 
competency-based adult literacy assessment measures an indi- 
vidual's performance against a predetermined standard of 
acceptable performance. Progress is based on actual perform- 
ance rather than on how well learners perform in comparison 
to others (Lytle and Wolfe 1989; Sticht 1990). 
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Competency-based education and assessment were developed 
in response to the need to assess adult literacy achievement 
within a functional framework. Because it recognizes the 
importance of grior learning and rewards what individuals can 
already do, it is more compatible for use with adults than 
standardized testing or the materials-based approach. Assess- 
ment is also frequent, providing learners with regular feedback 
and allowing them to advance when ready (Lytle and Wolfe 
1989). 

Despite its compatibility with adult education philosophy and 
practice, competency-based assessment also nas its critics. 
Because competency-based assessment usually takes place with- 
in the educational setting, it is still a test given under classroom 
conditions; thus a key theoretical concept of successful func- 
tioning in life roles is removed from the assessment process. 
Some critics also contend that, like the materials-based 
approach, competency-based assessment systems control and 
restrict teaching and learning (ibid.). 



Participatory Assessment 

Participatory assessment is a process that views assessment as 
much more than testing. Features of participatory assessment 
include a view of literacy as practices and critical reflection, the 
use of a broad range or strategies in assessment, and an active 
role for learners in the assessment process (BCEL 1990; Lytle 
and Wolfe 1989). Those advocating a participatory approach 
do so because of a belief that "learners, their characteristics, 
aspirations, backgrounds, and needs should be at the center of 
literacy instruction" (Fingeret and Jurmo 1989, p. 5). 

The following assumptions support the participatory assessment 
process: "the paramount purpose of assessment should be to 
ncjp the learner achieve his or her goals; what is assessed must 
reflect what the learner wishes or needs to accomplish; the pro- 
cess must build on the learner's experience and strengths rather 
than deficits; assessment is not something done to the learner; 
[and] it should not be externally imposed nor shrouded in 
mvstery, nor separated from what goes on in the regular course 
oncarning activity" (BCEL 1990, p. 7). 

Sometimes known as "alternative assessment approaches or 
methods" (BCEL 1990; Sticht 1990), elements of participatory 
assessment have been adopted by a number of adult literacy 
educators. The Adult Literacy Evaluation Project (ALEP) in 
Philadelphia is a project that includes many features of par- 
ticipatory assessment. This collaborative research project has 
developed alternatives to standardized tests and grade-level 
equivalences in measuring progress in literacy. The California 
Adult Learner Progress Evaluation Process, a joint program of 
the California State Libraries/California Literacy Campaign 
and the Educational Testing Service, also employs some partici- 
patory approaches to assessment. It uses forms developed for 
oint use by tutors and learners but that are written with the 
earner as the primary audience (Lytle and Wolfe 1989). 



Despite its congruence with many of the^ assumptions under- 
lying good adult education practice, participatory assessment is 
not without its critics. One question nas to do with whether the 
use of alternate forms of assessment-rather than standardized 
tests-leads to less demanding levels of achievement. Also, sole 
reliance on nonstandardized methods makes it difficult to make 
comparisons with other programs for the purpose of program 
evaluation (Sticht 1990). 

Conclusion 

Given the plethora of approaches and instruments^ available for 
assessing adult literacy learners, what should guide the deci- 



sions about which to use? Nurss (1989) suggests the following 
questions be considered in selecting assessment instruments and 
procedures for use in adult literacy: What is the purpose of the 
assessment?, Is the assessment instrument appropriate for use 
with adults?, How reliable, practical, and valid is the instru- 
ment?, Is the instrument culturally sensitive?, and Is there 
congruence between the instrument/approach and the 
instruction. 

According to Lytle and Wolfe (1989) "of prime importance 
seems to be the degree of congruence between particular 
approaches and a program's curricula and teaching practices" 
(p. 57). However, some interpret "the degree K congruence" 
to mean that both instruction and assessment should be 
standardized. Also, some question whether any single measure 
is capable of capturing the repertoire of skills and strategies an 
individual needs to accomplish a variety of literacy tasks. 

Because of the variety of learner goals and accomplishments, 
multiple methods of assessment seem logical. Such an 
approach provides learners, teachers, and other stakeholders 
with multiple views of learner accomplishments. 
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Questions To Ask When 
Evaluating Tests 

Lawrence M. Rudner, ERIC/AE 

The Standards for Educational and Psychological Testing 
established by the American Educational Research Association, 
the American Psychological Association, and the National 
Council on Measurement in Education, are intended to provide 
a comprehensive basis for evaluating tests. This Digest 
identifies the key standards applicable to most test evaluation 
situations. Sample questions are presented to help in your 
evaluations. 

Test Coverage and Use 

There must be a clear statement of recommended uses and a 
description of the population for which the test is intended. 

The principal question to ask when evaluating a test is 
whether it is appropriate for your intended purposes as well as 
your students. The use intended by the test developer must be 
justified by the publisher on technical grounds. You then need 
to evaluate your intended use against the publisher's intended 
use. Questions to ask: 

1. What are the intended uses of the test? What 
inteipretations does the publisher feel are appropriate? Are 
inappropriate applications identified? 

2. Who is the test designed for? What is the basis for 
considering whether the test applies to your students? 

Appropriate Samples for Test Validation and Norming 

The samples used for test validation and norming must be of 
adequate size and must be sufficiently representative to 
substantiate validity statements, to establish appropriate norms, 
and to support conclusions regarding the use of the instrument 
for the intended purpose. 

The individuals in the norming and validation samples 
should represent the group for which the test is intended in 
terms of age, experience and background. Questions to ask: 

1. How were the samples used in pilot testing, validation and 
norming chosen ? How is this sample related wyour 
student population? Were participation rates appropriate? 

2. Was the sample size large enough to develop stable 
estimates with minimal fluctuation due to sampling errors? 
Where statements are made concerning subgroups, are 
there enough teshtakers in each subgroup? 
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3. Do the difficulty levels of the test and criterion measures (if 
any) provide an adequate basis for validating and norming 
the instrument? Are there sufficient variations in test 
scores? 

Reliability 

The test is sufficiently reliable to permit stable estimates of 
individual ability. 

Fundamental to the evaluation of any instrument is the 
degree to which test scores are free from measurement error 
and are consistent from one occasion to another. Sources of 
measurement error, which include fatigue, nervousness, content 
sampling, answering mistakes, misinterpreting instructions and 
guessing, contribute to an individual's score and lower a test's 
reliability. 

Different types of reliability estimates should be used to 
estimate the contributions of different sources of measurement 
error. Inter-rater reliability coefficients provide estimates of 
errors due to inconsistencies in judgment between raters. 
Alternate-form reliability coefficients provide estimates of the 
extent to which individuals can be expected to rank the same on 
alternate forms of a test. Of primary interest are estimates of 
internal consistency which account for error due to content 
sampling, usually the largest single component of measurement 
error. Questions to ask: 

1. How have reliability estimates been computed? Have 
appropriate statistical methods been used? (e.g., Split half- 
reliability coefficients should not be used with speeded tests 
as they will produce artificially high estimates.) 

2. What are the reliabilities of the test for different groups of 
test-takers? How were they computed? 

3. Is the reliability sufficiently high to warrant using the test as 
a basis for decisions concerning individual students? 

Predictive Validity 

The test adequately predicts academic performance. 

In terms of an achievement test, predictive validity refers to 
the extent to which a test can be used to draw inferences 
regarding achievement. Empirical evidence in support of 
predictive validity must include a comparison of performance on 
the validated test against performance on outside criteria. A 
variety of measures are available, such as grades, class rank, 
other tests and teacher ratings. 

There are also several ways to demonstrate the relationship 
between the test being validated and subsequent performance. 
In addition to correlation coefficients, scatterplots, regression 
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equations and expectancy tables should be provided. Questions 
to ask: 

1. What criterion measure has been used to evaluate validity? 
What is the rationale for choosing this measure? 

2. Is the distribution of scores on the criterion measure 
adequate? 

3. What is the overall predictive accuracy of the test? How 
accurate are predictions for individuals whose scores are 
close to cut-points of interest? 

Content Validity 

Content validity refers to the extent to which the test questions 
represent the skills in the specified subject area. 

Content validity is often evaluated by examining the plan 
and procedures used in test construction. Did the test 
development procedure follows rational approach that ensures 
appropriate content? Did the process ensure that the collection 
of items would represent appropriate skills? Other questions to 
ask: 

1. Is there a clear statement of the universe of skills 
represented by the test? What research was conducted to 
determine desired test content and/or evaluate content? 

2. What was the composition of expert panels used in content 
validation? How were judgments elicited? 

3. How similar is this content to the content you are interested 
in testing? 

Construct Validity 

The test measures the "right" psychological constructs. 

Intelligence, self-esteem and creativity are examples of 
such psychological traits. Evidence in support of construct 
validity can take many forms. One approach is to demonstrate 
that the items within a measure are inter-related and therefore 
measure a single construct. Inter-item correlation and factor 
analysis are often used to demonstrate relationships among the 
items. Another approach is to demonstrate that the test behaves 
as one would expect a measure of the construct to behave. For 
example, one might expect a measure of creativity to show a 
greater correlation with a measure of artistic ability than with a 
measure of scholastic achievement. Questions to ask: 

1. Is the conceptual framework for each tested construct clear 
and well founded? What is the basis for concluding that the 
construct is related to the purposes of the test? 

2. Does the framework provide a basis for testable hypotheses 
concerning the construct? Are these hypotheses supported 
by empirical data? 

Test Administration 

Detailed and clear instructions outline appropriate test 
administration procedures, 

Statements concerning test validity and the accuracy of the 
norms can only generalize to testing situations which replicate 
the conditions used to establish validity and obtain normative 
data. Test administrators need detailed and clear instructions to 
replicate these conditions. 

All test administration specifications, including instructions 
to test takers, time limits, use of reference materials and 
calculators, lighting, equipment, seating, monitoring, room 



requirements, testing sequence, dx4 time of day, should be full 
described. Questions to ask: 

L Will text administrators under stana precisely what is 

expected of them? 
2. Do the test administration procedures replicate the 

conditions under which che test was validated and normed? 

Are these procedures standardized? 

Tcsl Reporting 

7hi? methods used to report test results, including scaled scores, 
subtests results and combined test results, are described fully 
along with the rationale for each method. 

Test results should be presented in a manner that will help 
schools, teachers and students to make decisions that are 
consistent \n*h appropriate uses of the test. Help should be 
available for interpreting and using the test results. Questions 
to ask: 

1. How are test results reported? Are the scales used in 
reporting results conducive to proper test use? 

2. What materials and resources are available to aid in 
interpreting test results? 

Test and Item Bias 

The test is not biased or offer sive with regard to race, sex, 
native language, ethnic origin, geographic region or other 
factors. 

Test developers are expected to exhibit a sensitivity to the 
demographic characteristics of test-takers. Steps can be taken 
during test development, validation, standardization and 
documentation to minimize the influence of cultural factors on 
individual test scores. These steps may include evaluating items 
for offensiveness and cultural dependency, using statistics to 
identify differential item difficulty, and examining the predictive 
validity for different groups. 

Te c ts are not expected to yield equivalent mean scores 
across population groups. Rather, tests should yield the same 
scores and predict the same likelihood of success for individual 
test-takers of the same ability, regardless of group membership. 
Questions to ask: 

1. Were the items analyzed statistically for possible bias? 
What method(s) was used? How were items selected for 
inclusion in the final version of the test? 

2. Was the test analyzed for differential validity across 
groups? How was this analysis conducted? 

3. Was the test analyzed to determine the English language 
proficiency required of tesnakers? Should the test be used 
with non-native speakers of English? 
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ASSESSMENT TOOLS AS REPORTED BY RESPONDENTS 



Adult Basic Learning Examination (ABLE) 

Adult Placement Indicator (API) 

ALS 

ALT 

ASSET 

Bader Reading and Language Inventory 
Botel Reading Inventory 

Brigance Diagnostic Comprehension Inventory of Basic Skills 
Classroom Reading Inventory (CRI) 

Comprehensive Adult Student Assessment System (CASAS) 
Diagnostic Analysis of Reading Errors (DARE) 
Diagnostic Spelling Potential Test 
Doran Diagnostic Inventory 

English as a Second Language Oral Assessment (ESLOA) 

Gates-MacGinitie Reading Tests 

Kottmeyer 

Language Experience Activity 

Language Inventory 

Laubach Materials 

Literacy Assessment Survey (LAS) 

Macomb Placement Packet 

MEAP 

Merrill Linguistic Reading Program 
Methodist Women's Reading Test 
Metropolitan Achievement Test 
MOTT 

Oral Reading, Vision, Auditory Discrimination 
Pre-GED 

Portfolio Assessment 
RCDA Paragraphs 

Reading Evaluation Adult Diagnosis (READ) 
REVRAC 

Schonell Graded Word Reading Test 
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Slosson Oral Reading Test (SORT) 
Stanford Diagnostic Reading Test 
Tests of Adult Basic Education (TABE) 
Wepman 
Where To Start 

Wide Range Achievement Test (WRAT) 
Woodcock Reading Mastery Test 
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